Augmented reality item obscuring

ABSTRACT

Augmented reality (AR) technology can be used to perform a real-time overlay on an image displayed on a user device. A user device can receive image data of an area comprising an item from the real-world. Based on the image data, an identity and a value of the item can be obtained by the user device. If the value of the item is greater than a pre-determined value for the item or a category of the item, then the user device can obscure in real-time on the displayed image the item or category of the item having the value that is greater than the pre-determined value.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a non-provisional of and claims priority to U.S.Provisional Application No. 62/641,790, filed on Mar. 12, 2018, entitled“AUGMENTED REALITY ITEM OBSCURING,” which is hereby incorporated byreference in its entirety for all purposes.

BACKGROUND

Augmented reality (AR) technology is used to add virtual objects to adigital display to enhance user experience. For example, when a useruses a mobile device equipped with AR technology to look at stars in thesky, the application on the mobile device can add lines to connect starsin a constellation and can add a text identifying the variousconstellations. Thus, AR technology can be used to add content to areal-world image to enhance user experience.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A shows an embodiment where a user device receives image data of areal-world that includes two cars.

FIG. 1B show an embodiment where one or more items having a valuegreater than a user's budget is obscured on a display.

FIG. 2 shows a flowchart for using AR technology to perform a real-timeoverlay on an image displayed on a user device.

FIG. 3 shows another flowchart for using AR technology to perform areal-time overlay on an image displayed on a user device.

FIG. 4 is a block diagram illustrating an overview of devices on whichsome implementations can operate.

FIG. 5 is a block diagram illustrating an overview of an environment inwhich some implementations can operate.

FIG. 6 is a block diagram illustrating components which, in someimplementations, can be used in a system employing the disclosedtechnology.

The techniques introduced here may be better understood by referring tothe following Detailed Description in conjunction with the accompanyingdrawings, in which like reference numerals indicate identical orfunctionally similar elements. Moreover, while the technology isamenable to various modifications and alternative forms, specificembodiments have been shown by way of example in the drawings and aredescribed in detail below. The intention, however, is not to limit thetechnology to the particular embodiments described. On the contrary, thetechnology is intended to cover all modifications, equivalents, andalternatives falling within the scope of the technology as defined bythe appended claims.

DETAILED DESCRIPTION

AR technology is generally used to add a layer of information to adisplay of a real-world image. Thus, AR technology generally increasesthe amount of information that is presented to the user. The embodimentsdescribed in this patent document use AR technology to enhance userexperience by reducing the information presented about the real world.In some embodiments, AR technology is used to obscure, hide, or removecertain content from a display to provide a personalized experience to auser for various reasons. For example, the system can obscure itemsbased on the user's financial budget (e.g., certain items are obscuredto help the user stay within a budget), health goals (e.g., certainfoods are obscured at the grocery store to help the user stick to adiet), environmental or safety goals (e.g., certain items are removedthat do not meet safety or environmental standards), or other userpreferences.

When shopping on the internet, vendors can offer users a way to filtergoods displayed based on a price limit. However, when shopping in thephysical world, a user cannot similarly filter his or her choices basedon a price of the item or some other criteria. Thus, shoppers can besusceptible to purchasing items outside of their budget or purchasingitems that do not meet the user's preferences. Shoppers can also succumbto up-sell tactics from a salesperson. This is especially relevant whenshopping for big ticket items such as cars, furniture, or electronics.

FIGS. 1A and 1B show an embodiment where one or more items having avalue greater than a user's budget is obscured, or hid, or removed on adisplay. FIG. 1A shows a user device 102 receiving image data of areal-world that includes two cars 104, 106 parked in a car dealershipusing a user device 102. The user device 102 can include a phone, atablet, a digital eyeglass device, or other wearable technology such asa helmet styled device with a visor or a digital contact lens. The userdevice 102 can receive image data of an area 108 that includes one ormore items from the real world. The image data is wirelessly sent by theuser device 102 to a server (not shown) that can process the image datato determine the identities and values of the one or more items. In theexample shown in FIG. 1, the server determines the identities, values,or other information (e.g., safety, emissions) of the two cars 104, 106and sends the identity and value information back to the user device102. The user device 102 and the server can send information to eachother using Wi-Fi or cellular signals.

In some embodiments, when shopping for an item, a user can set a maximumpre-determined value for the item or category of item, such as a budgetfor an item so that the user can determine whether one or more itemsfall within the user's budget for purchase, or an application can obtainthis information from a budget service that is accessed with the user'sprior permission. Using the example of FIGS. 1A and 1B, in someembodiments, the user can set a budget for a category, such as a car, orthe user can set a budget for an item such as a specific year and modelof the car. When the user device obtains the value of the car from theserver, the user device can determine whether the value of the car isgreater than the user set maximum pre-determined value. In FIG. 1B, theuser device 102 is shown as a phone that displays the image of the area108 comprising two cars 104, 106 and obscures in real-time on thedisplayed image the car 104 that has a price greater than the user'sbudget. In some embodiments, an image displayed by the user device canbe a moving image, such as a video, so that an item can be obscured inreal-time on the moving image. In some embodiments, the user device canbe a digital eyeglass device or a tablet.

In some embodiments, a user device can obscure one or more items byblurring the item, graying out the item, or obscuring source-identifyinginformation of the item. Source-identifying information can include agrill guard on a car, an emblem, or a shape. The obscuring or hiding ofone or more items that are outside of the user's budget can betterindicate to the user what item or items to choose from to stay within aprescribed budget or other preference.

Various methods can be used to obscure or hide an item. Such methods maybe used depending on factors such as the environment of the user, theuser's preferences, and the computational capabilities of the softwareand hardware. Examples of methods of obscuring an item can include:

-   -   obscuring by inserting an opaque polygon into the image or        moving video image (the polygon can be white, black or any        color),    -   obscuring by inserting a semi-transparent polygon into the image        or moving video image (the polygon can be white, black or any        color tint),    -   obscuring by any of the two aforementioned methods and adding        text or icons that tells the user which object was recognized        and why it is obscured (e.g. item is more expensive than the set        budget),    -   obscuring by pixelating an object, and    -   obscuring by reconstructing the inferred background of an        object, giving the impression to the user that the object is not        there or that it is invisible.

A benefit of obscuring items for purchase is that it allows the user tomake better decisions to meet financial or other goals. Another benefitof obscuring items is that it can help dissuade the user from purchasingmore expensive items that exceed the user's budget.

In some embodiments, the area of the augmented reality device that isobscured can be used to recommend other products. In an example,assuming that a sporty car outside the user's budget was blurred, therecommendation area can mention a similar product within the user'sbudget such as “If you are looking for a sporty car, we recommend an XYZcar.”

While FIGS. 1A and 1B disclose an embodiment where a single user devicecan operate with a server, in some embodiments, a user device caninclude two devices, such as a mobile device and a display device, thatcan operate with each other and a server to perform the real-timeoverlay on an image displayed on the display device. As an example, adisplay device, such as a digital eyeglass device, can receive imagedata from a camera and can send the image data to a mobile device, suchas a phone or a tablet, that sends the image data to the server. In someembodiments, the server can determine the identity and one or morevalues or characteristics of the item. When the mobile device receivesthe one or more values from the server, the mobile device can determinewhether an item should be obscured by comparing a value to apre-determined value as described in this patent document. In otherembodiments, the server determines which items should be obscured andthe mobile device is simply used as a proxy. If the mobile device (orserver) determines that an item should be obscured, the display device,such as a digital eyeglass device, that display the image can obscure inreal-time on the displayed image the item. In some embodiments, themobile device may transmit to the display device information about thearea to obscure. In some other embodiments, the display device and themobile device can work together to form an object understanding of thescene, and after retrieving price and other identifying informationabout the items, the mobile device can transmit to the display devicethe shared item identifier of the item to obscure. The mobile device andthe display device can communicate with each other using wirelesstechnology, for example, Wi-Fi or Bluetooth.

In some embodiments, a maximum pre-determined value can be set by a userin decision-making contexts other than financial decisions. For example,a user device can obscure one or more items on a supermarket shelfhaving a number or calories or an amount of sugar that is greater than amaximum pre-determined value set by a user or recommended by a healthapplication or service used by the user. Based on the image datareceived by the user device and sent to the server, the server canidentify and can provide to the user device the nutritional value of theone or more items. The user device can obscure in real-time on thedisplayed image the one or more items on the supermarket shelf if theuser device determines that a nutritional value of the one or more itemsis out of range compared to a pre-determined range set by a user orrecommended by a health application or service used by the user. Abenefit of obscuring some food items is that it allows the user to makepersonalized and better health-related decisions.

In some embodiments, a user can set a minimum pre-determined value forthe item or category of item having certain minimum characteristics, forexample, a fuel economy of a car, the safety rating of a car, anutritional value of a food item, a product rating, or date ofmanufacture of an item. Based on the image data received by the userdevice and sent to the server, the server can identify and can provideto the user device the minimum characteristics values of the one or moreitems. The user device can obscure in real-time on the displayed imageone or more items if the user device determines that the values of theminimum characteristics of the one or more items is less than a minimumpre-determined value set by a user. In some embodiments, a user can seta budget as a minimum characteristic so that one or more items having avalue less than a minimum budget can be obscured to allow a user to viewa displayed item having a value that is greater than the minimum setbudget.

The minimum or maximum pre-determined value can be set by a user. Forexample, a user can set a safety rating of 4.0 or above or a fueleconomy of 30 miles per gallon or above as a preferred minimumpre-determined value. In some embodiments, a user may set both a minimumand maximum pre-determined value for an item characteristic so that oneor more items that fall outside a pre-determined range can be obscured.

The pre-determined values and designations to identify whether thepre-determined values are maximum or minimum values can be stored on anapplication on the user device or it can be stored on a server to beobtained by the user device. In some embodiments, a server can determinewhether a value of an item is greater than a maximum pre-determinedvalue or the server can determine whether a value of an item is lessthan a minimum pre-determined value. When a server determines that anitem should be obscured, the server can send to the user device theidentity of an item to be obscured.

From a still or live video image, the server's analysis can identifydistinct items or objects as the first part of the analysis of theimagery. Each object can be given an ID that is shared between thevarious computing devices, such as the server and user devices. In someembodiments, if the device that is receiving the image data about thescene has enough computing power, an instruction can be transmitted tothe image data receiving device to obscure one or more object with thecorresponding specific IDs. In some embodiments, if a display devicereceiving the image data about the scene lacks computing power, a mobiledevice can receive the one or more object IDs to be obscured from theserver and the mobile device can translate the one or more object IDsinformation into one or more areas to obscure and transmit thatinstruction to the display device.

In some embodiments, before a user device can send image data to aserver to obtain information about one or more items in the image data,the user device can determine a location of the user device andascertain whether the location is associated with a place that performstransactions (e.g., a place that a user can make a purchase). Using theexample provided in FIG. 1A, in some embodiments, the user device orserver can determine a location of the user device, using, for example,geo-coordinates or latitude and longitude, and ascertain that theparticular location is associated with a “Downtown Car Dealership” thatsells cars. As described in this patent document, the user device cansubsequently receive image data, obtain identities and values of one ormore items from a server, compare the value with a pre-determined value,and obscure one or more items based on the comparison. In some otherembodiments, user device can provide its location information to aserver and the server can ascertain whether a user device is located ata place that a performs transactions. In some embodiments, anidentification of one or more objects from an image data and thesubsequent determination of which objects to obscure may be performedwithout obtaining location information.

In some embodiments, the user device or server can providerecommendations to the user in the portion of the augmented realitydevice with obscured information. The recommendations can includerecommendations specific to what is obscured. For example, assuming thata sporty car outside the user's budget was blurred, the recommendationarea can mention a similar product within the user's budget such as “Ifyou are looking for a sporty car, we recommend an XYZ car.”

Those skilled in the art will appreciate that the components illustratedin FIGS. 4-6 described below, and in each of the flow diagrams discussedbelow, may be altered in a variety of ways. For example, the order ofthe logic may be rearranged, substeps may be performed in parallel,illustrated logic may be omitted, other logic may be included, etc. Insome implementations, one or more of the components described below canexecute one or more of the processes described below.

FIG. 2 shows a flowchart for using AR technology to perform a real-timeoverlay on an image displayed on a user device. At a receiving operation202, a user device receives image data of an area comprising an itemfrom the real-world. In some embodiments, prior to the receivingoperation 202, the user device or server can determine the user device'slocation and the user device or a server can ascertain that the locationof the user device is associated with a place that performstransactions.

At the obtaining operation 204, a user device obtains, based on theimage data, an identity and a value of the item. The image data obtainedby the user device is sent to a server that determines and sends theidentity and value of the item to the user device.

At the determining operation 206, the user device or a server candetermine that the value of the item is greater than a pre-determinedvalue for the item or a category of the item. The pre-determined valuecan be a maximum user preferred value and can include, for example, abudget for the item or the category of the item. The pre-determinedvalue can be set by a user and can be stored on an application on a userdevice or it can be retrieved by the user device from a server thatstores the pre-determined value.

At the displaying operation 208, the user device displays the image ofthe area comprising the item. At the obscuring operation 210, the userdevice obscures, in real-time on the displayed image, the item orcategory of the item having the value that is greater than thepre-determined value. An item can be obscured by blurring the item,graying out the item, or obscuring source-identifying information of theitem. In some embodiments, the item is hidden. In other embodiments, asymbol indicating a characteristic is placed on the item (e.g., dollarsigns are put over an item that is outside of a user's budget). In someembodiments, the area that is obscured can provide recommendations onsubstitute or alternative products to the obscured item (e.g., “if youare looking for a set of binoculars, we recommend XYZ brand, model ZZ.”

The user device can be a phone, a tablet, a digital eyeglass device, orother wearable device. In some embodiments, a user device can performthe operations associated with FIG. 2 in a continuous routine to allowthe user device to perform real-time overlay on moving images.

In some embodiments, a real-time overlay can be performed on an imagedisplayed on a user device based on user defined minimumcharacteristics. As an example, a minimum characteristic can include afuel economy of a car, a nutritional value of a food item, a productrating, or date of manufacture of an item. Based on an image data sentto the server, the user device can receive a second value of the item.The user device or the server can determine whether the second value ofthe item is less than a second pre-determined value for the item or thecategory of the item. The second-predetermined value is a minimum userpreferred value. As an implementation example, the second pre-determinedvalue can also be set by a user and can be stored on an application on auser device or it can be retrieved by the user device from a server thatstores the second pre-determined value. The user device can obscure, inreal-time on the displayed image, the item or category of the itemhaving the second value that is less than the second pre-determinedvalue.

A user device can run an application to access information from otherapplications located on the user device or on a server. For example, auser device application with an Application Programming Interface (API)can obtain from a map application location of a user device orinformation that the user device is located at or near a store, a cardealership, a mall, or a restaurant. The user device application canalso use another API to obtain from a server the identities and valuesof items, such as cars.

FIG. 3 shows another flowchart for using AR technology to perform areal-time overlay on an image displayed on a user device. At thereceiving operation 302, a user device can receive image data of an areacomprising an item from the real-world. At the obtaining operation 304,the user device can obtain from a server an identity and a value of theitem based on the image data. At the displaying operation 306, the userdevice can display the image of the area comprising the item.

At the first determining operation 308, a user device or a server candetermine whether an inequality indicator that compares the value to apre-determined value is a greater than symbol. If the inequalityindicator is a greater than symbol then the process moves to the seconddetermining operation 310 where a user device or a server can determinethat the value of the item is greater than the pre-determined value forthe item or the category of the item. At the obscuring operation 312,the user device can obscure, in real-time on the displayed image, theitem or category of the item having the value that is greater than thepre-determined value.

At the third determining operation 314, a user device or a server candetermine whether the inequality indicator that compares the value to apre-determined value is a less than symbol. If the inequality indicatoris a less than symbol, then the process moves to the fourth determiningoperation 316 where a user device or a server can determine that thevalue of the item is less than the pre-determined value for the item orthe category of the item. At the obscuring operation 318, the userdevice can obscure, in real-time on the displayed image, the item orcategory of the item having the value that is less than thepre-determined value.

The inequality indicator can be provided to the user device from aserver or it can be stored on the user device. In some embodiments, auser device can perform the operations associated with FIG. 3 in acontinuous routine to allow the user device to perform real-time overlayon moving images.

Several implementations are discussed below in more detail in referenceto the figures. Turning now to the figures, FIG. 4 is a block diagramillustrating an overview of devices on which some implementations of thedisclosed technology can operate. The devices can comprise hardwarecomponents of a device 400 that send image data from an area thatincludes one or more items from a real-world, receive the identity andcharacteristics of one or more items, and obscure one or items based onpre-determined values. Device 400 can include one or more input devices420 that provide input to a CPU (processor) or GPU 410, notifying it ofactions. The actions are typically mediated by a hardware controllerthat interprets the signals received from the input device andcommunicates the information to the CPU or GPU 410 using a communicationprotocol. Input devices 420 include, for example, a mouse, a keyboard, atouchscreen, an infrared sensor, a touchpad, a wearable input device, acamera- or image-based input device, a microphone, or other user inputdevices.

CPU or GPU 410 can be a single processing unit or multiple processingunits in a device or distributed across multiple devices. CPU or GPU 410can be coupled to other hardware devices, for example, with the use of abus, such as a PCI bus or SCSI bus. The CPU or GPU 410 can communicatewith a hardware controller for devices, such as for a display 430.Display 430 can be used to display text and graphics. In some examples,display 430 provides graphical and textual visual feedback to a user. Insome implementations, display 430 includes the input device as part ofthe display, such as when the input device is a touchscreen or isequipped with an eye direction monitoring system. In someimplementations, the display is separate from the input device. Examplesof display devices are: an LCD display screen; an LED display screen; aprojected, three-dimensional, holographic, or augmented reality display(such as a heads-up display device or a head-mounted device); and so on.Other I/O devices 440 can also be coupled to the processor, such as anetwork card, video card, audio card, USB, FireWire or other externaldevice, camera, printer, speakers, CD-ROM drive, DVD drive, disk drive,or Blu-Ray device.

In some implementations, the device 400 also includes a communicationdevice capable of communicating wirelessly or wire-based with a networknode. The communication device can communicate with another device or aserver through a network using, for example, TCP/IP protocols. Device400 can utilize the communication device to distribute operations acrossmultiple network devices.

The CPU or GPU 410 can have access to a memory 450. A memory includesone or more of various hardware devices for volatile and non-volatilestorage, and can include both read-only and writable memory. Forexample, a memory can comprise random access memory (RAM), CPUregisters, read-only memory (ROM), and writable non-volatile memory,such as flash memory, hard drives, floppy disks, CDs, DVDs, magneticstorage devices, tape drives, device buffers, and so forth. A memory isnot a propagating signal divorced from underlying hardware; a memory isthus non-transitory. Memory 450 can include program memory 460 thatstores programs and software, such as an operating system 462, augmentedreality platform 464, and other application programs 466. Memory 450 canalso include data memory 470 that can include pre-determined values foran item or a category of the item, designations to identify whether thepre-determined values are maximum or minimum values, settings, orpreferences, etc., which can be provided to the program memory 460 orany element of the device 400.

Some implementations can be operational with numerous other generalpurpose or special purpose computing system environments orconfigurations. Examples of well-known computing systems, environments,and/or configurations that may be suitable for use with the technologyinclude, but are not limited to, personal computers, server computers,handheld or laptop devices, cellular telephones, wearable electronics,gaming consoles, tablet devices, multiprocessor systems,microprocessor-based systems, set-top boxes, programmable consumerelectronics, network PCs, minicomputers, mainframe computers,distributed computing environments that include any of the above systemsor devices, or the like.

FIG. 5 is a block diagram illustrating an overview of an environment 500in which some implementations of the disclosed technology can operate.Environment 500 can include one or more client computing devices 505A-E,examples of which can include device 400. Client computing devices 505can operate in a networked environment using logical connections throughnetwork 530 to one or more remote computers, such as a server computingdevice 510.

In some implementations, server computing device 510 can be an edgeserver that receives client requests and coordinates fulfillment ofthose requests through other servers, such as servers 520A-C. Servercomputing devices 510 and 520 can comprise computing systems, such asdevice 400. Though each server computing device 510 and 520 is displayedlogically as a single server, server computing devices can each be adistributed computing environment encompassing multiple computingdevices located at the same or at geographically disparate physicallocations. In some implementations, each server computing device 520corresponds to a group of servers.

Client computing devices 505 and server computing devices 510 and 520can each act as a server or client to other server/client devices.Server 510 can connect to a database 515. Servers 520A-C can eachconnect to a corresponding database 525A-C. As discussed above, eachserver 520 can correspond to a group of servers, and each of theseservers can share a database or can have their own database. Databases515 and 525 can warehouse (e.g., store) information such aspre-determined values for an item or a category of the item,designations to identify whether the pre-determined values are maximumor minimum values, identities and characteristics of items, or locationinformation. Though databases 515 and 525 are displayed logically assingle units, databases 515 and 525 can each be a distributed computingenvironment encompassing multiple computing devices, can be locatedwithin their corresponding server, or can be located at the same or atgeographically disparate physical locations.

Network 530 can be a local area network (LAN) or a wide area network(WAN), but can also be other wired or wireless networks. Network 530 maybe the Internet or some other public or private network. Clientcomputing devices 505 can be connected to network 530 through a networkinterface, such as by wired or wireless communication. While theconnections between server 510 and servers 520 are shown as separateconnections, these connections can be any kind of local, wide area,wired, or wireless network, including network 530 or a separate publicor private network.

FIG. 6 is a block diagram illustrating components 600 which, in someimplementations, can be used in a system employing the disclosedtechnology. The components 600 include hardware 602, general software620, and specialized components 640. As discussed above, a systemimplementing the disclosed technology can use various hardware,including processing units 604 (e.g., CPUs, GPUs, APUs, etc.), workingmemory 606, storage memory 608, and input and output devices 610.Components 600 can be implemented in a client computing device such asclient computing devices 505 or on a server computing device, such asserver computing device 510 or 520.

General software 620 can include various applications, including anoperating system 622, local programs 624, and a basic input outputsystem (BIOS) 626. Specialized components 640 can be subcomponents of ageneral software application 620, such as local programs 624.Specialized components 640 can include an augmented reality platformthat includes an image data receiving module 644, image data processingmodule 646, item obscuring module 648, location determination module650, and components that can be used for transferring data andcontrolling the specialized components, such as interface 642. In someimplementations, components 600 can be in a computing system that isdistributed across multiple computing devices or can be an interface toa server-based application executing one or more of specializedcomponents 640.

Image data receiving module 644 receives image data of an area thatincludes one or more items from a real world. The image data receivingmodule 644 can send the received image data to the image data processingmodule 646 where the image data is be processed to obtain an identityand other characteristics of the items in the image data. Thecharacteristics of the items may include, for example, a cost of theitem or other characteristics relevant to the identified items. Anexample of other characteristics can include fuel economy if theidentified item is a car, or a nutritional value if the identified itemis a food item, or a consumer rating if the item is television.

Item obscuring module 648 can compare the value of an item included inthe image data with a pre-determined value. In some implementations, thepre-determined value can be a maximum user defined value so that theitem obscuring module 648 can obscure in real-time on the displayedimage the item or category of the item that has a value that is greaterthan the pre-determined value. In some other implementations, thepre-determined value can be a minimum user defined value so that theitem obscuring module 648 can obscure in real-time on the displayedimage the item or category of the item that has a value that is lessthan the pre-determined value. The item obscuring module 648 can obscurean item by blurring the item, graying out the item, or obscuringsource-identifying information of the item.

Item obscuring module 648 can also determine how to compare the value ofthe item with the pre-determined value. In some implementations, aninequality indicator, such as a greater than or less than symbol, can beprovided by the item obscuring module 648 based on a user selection ofhow an item's characteristics can be compared to a user setpre-determined value. For example, if a user wants to purchase a car,the user can set a maximum budget of $20,000, or the value can resultfrom a budgeting software used by the user. The item obscuring module648 can store or retrieve from a database the maximum budget with a“greater than” symbol to indicate that items over $20,000 are obscuredby the item obscuring module 648. As another example, a user can set apre-determined range of values so that the item obscuring module 648 canobscure one or more items that fall out of the range of the maximum andminimum values that may be set by the user. In some embodiments, itemscan be categorically obscured or hidden. For example, users can set apreference of obscuring all products made by a certain company.

The location determination module 650 can determine a location of a userdevice and can ascertain whether the location of the user device isassociated with a place that performs transactions, such as financial orpurchasing transactions. In some implementations, if the locationdetermination module 650 determines that a user device is located at ornear a transaction, the location determination module 650 can initiate aprocess of receiving image data to be further processed as described inthis patent document.

Those skilled in the art will appreciate that the components illustratedin FIGS. 4-6 described above, and in each of the flow diagrams discussedbelow, may be altered in a variety of ways. For example, the order ofthe logic may be rearranged, substeps may be performed in parallel,illustrated logic may be omitted, other logic may be included, etc. Insome implementations, one or more of the components described above canexecute one or more of the processes described below.

CONCLUSION

Unless the context clearly requires otherwise, throughout thedescription and the claims, the words “comprise,” “comprising,” and thelike are to be construed in an inclusive sense, as opposed to anexclusive or exhaustive sense; that is to say, in the sense of“including, but not limited to.” As used herein, the terms “connected,”“coupled,” or any variant thereof means any connection or coupling,either direct or indirect, between two or more elements; the coupling orconnection between the elements can be physical, logical, or acombination thereof. Additionally, the words “herein,” “above,” “below,”and words of similar import, when used in this application, refer tothis application as a whole and not to any particular portions of thisapplication. Where the context permits, words in the above DetailedDescription using the singular or plural number may also include theplural or singular number respectively. The word “or,” in reference to alist of two or more items, covers all of the following interpretationsof the word: any of the items in the list, all of the items in the list,and any combination of the items in the list.

Several implementations of the disclosed technology are described abovein reference to the figures. The computing devices on which thedescribed technology may be implemented can include one or more centralprocessing units, memory, input devices (e.g., keyboards and pointingdevices), output devices (e.g., display devices), storage devices (e.g.,disk drives), and network devices (e.g., network interfaces). The memoryand storage devices are computer-readable storage media that can storeinstructions that implement at least portions of the describedtechnology. In addition, the data structures and message structures canbe stored or transmitted via a data transmission medium, such as asignal on a communications link. Various communications links can beused, such as the Internet, a local area network, a wide area network,or a point-to-point dial-up connection. Thus, computer-readable mediacan comprise computer-readable storage media (e.g., “non-transitory”media) and computer-readable transmission media.

As used herein, being above a threshold means that a value for an itemunder comparison is above a specified other value, that an item undercomparison is among a certain specified number of items with the largestvalue, or that an item under comparison has a value within a specifiedtop percentage value. As used herein, being below a threshold means thata value for an item under comparison is below a specified other value,that an item under comparison is among a certain specified number ofitems with the smallest value, or that an item under comparison has avalue within a specified bottom percentage value. As used herein, beingwithin a threshold means that a value for an item under comparison isbetween two specified other values, that an item under comparison isamong a middle specified number of items, or that an item undercomparison has a value within a middle specified percentage range.

As used herein, the word “or” refers to any possible permutation of aset of items. For example, the phrase “A, B, or C” refers to at leastone of A, B, C, or any combination thereof, such as any of: A; B; C; Aand B; A and C; B and C; A, B, and C; or multiple of any item, such as Aand A; B, B, and C; A, A, B, C, and C; etc.

The above Detailed Description of examples of the technology is notintended to be exhaustive or to limit the technology to the precise formdisclosed above. While specific examples for the technology aredescribed above for illustrative purposes, various equivalentmodifications are possible within the scope of the technology. Forexample, while processes or blocks are presented in a given order,alternative implementations may perform routines having steps, or employsystems having blocks, in a different order, and some processes orblocks may be deleted, moved, added, subdivided, combined, and/ormodified to provide alternative or subcombinations. Each of theseprocesses or blocks may be implemented in a variety of different ways.Also, while processes or blocks are at times shown as being performed inseries, these processes or blocks may instead be performed orimplemented in parallel, or may be performed at different times. Furtherany specific numbers noted herein are only examples: alternativeimplementations may employ differing values or ranges.

The teachings of the technology provided herein can be applied to othersystems, not necessarily the system described above. The elements andacts of the various examples described above can be combined to providefurther implementations of the technology. Some alternativeimplementations of the technology may include not only additionalelements to those implementations noted above, but also may includefewer elements.

These and other changes can be made to the technology in light of theabove Detailed Description. While the above description describescertain examples of the technology, and describes the best modecontemplated, no matter how detailed the above appears in text, thetechnology can be practiced in many ways. Details of the system may varyconsiderably in its specific implementation, while still beingencompassed by the technology disclosed herein. As noted above,particular terminology used when describing certain features or aspectsof the technology should not be taken to imply that the terminology isbeing redefined herein to be restricted to any specific characteristics,features, or aspects of the technology with which that terminology isassociated. In general, the terms used in the following claims shouldnot be construed to limit the technology to the specific examplesdisclosed in the specification, unless the above Detailed Descriptionsection explicitly defines such terms. Accordingly, the actual scope ofthe technology encompasses not only the disclosed examples, but also allequivalent ways of practicing or implementing the technology under theclaims.

To reduce the number of claims, certain aspects of the technology arepresented below in certain claim forms, but the applicant contemplatesthe various aspects of the technology in any number of claim forms. Forexample, while only one aspect of the technology is recited as acomputer-readable medium claim, other aspects may likewise be embodiedas a computer-readable medium claim, or in other forms, such as beingembodied in a means-plus-function claim. Any claims intended to betreated under 35 U.S.C. § 112(f) will begin with the words “means for”,but use of the term “for” in any other context is not intended to invoketreatment under 35 U.S.C. § 112(f). Accordingly, the applicant reservesthe right to pursue additional claims after filing this application topursue such additional claim forms, in either this application or in acontinuing application.

I claim:
 1. A method of performing a real-time overlay on a displayedimage, the method comprising: receiving, by a user device, image data ofan area comprising an item from the real-world; obtaining, based on theimage data, an identity and a value of the item; determining that thevalue of the item is greater than a pre-determined value for the item ora category of the item; displaying on the user device the image of thearea comprising the item; and obscuring, in real-time on the displayedimage, the item or category of the item having the value that is greaterthan the pre-determined value.
 2. The method of claim 1, wherein theitem is obscured by any one of blurring the item, graying out the item,and obscuring source-identifying information of the item.
 3. The methodof claim 1, further comprising: obtaining, based on the image data, asecond value of the item; determining that the second value of the itemis less than a second pre-determined value for the item or the categoryof the item; and obscuring, in real-time on the displayed image, theitem or category of the item having the second value that is less thanthe second pre-determined value.
 4. The method of claim 3, wherein thepre-determined value is a maximum user defined value, and thesecond-predetermined value is a minimum user defined value.
 5. Themethod of claim 3, wherein the pre-determined value and the secondpre-determined value for the item are set by the user.
 6. The method ofclaim 1, wherein the pre-determined value includes a budget for the itemor the category of the item.
 7. The method of claim 1, wherein theobtaining of the identity and the value of the item comprises: sendingthe image data to a server, wherein server processes the image data todetermine the identity and the value of the item; and receiving, fromthe server, the identity and the value of the item.
 8. The method ofclaim 1, wherein prior to receiving the image data by the user device:determining a location of the user device; and ascertaining that thelocation of user device is associated with a place that performstransactions.
 9. The method of claim 1, wherein the user device includesany one of a phone, a tablet, and a digital eyeglass device.
 10. Themethod of claim 1, wherein the image includes a moving image.
 11. Amethod of performing a real-time overlay on a displayed image, themethod comprising: receiving, by a user device, image data of an areacomprising an item from the real-world; obtaining, based on the imagedata, an identity and a value of the item; displaying on the user devicethe image of the area comprising the item; in response to determiningthat an inequality indicator that compares the value to a pre-determinedvalue is a greater than symbol: determining that the value of the itemis greater than the pre-determined value for the item or a category ofthe item, obscuring, in real-time on the displayed image, the item orcategory of the item having the value that is greater than thepre-determined value; and in response to determining that the inequalityindicator that compares the value to the pre-determined value is a lessthan symbol: determining that the value of the item is less than thepre-determined value for the item or the category of the item,obscuring, in real-time on the displayed image, the item or category ofthe item having the value that is less than the pre-determined value.12. The method of claim 1, further comprising indicating to the userwhich item is obscured and why it is obscured.
 13. The method of claim11, wherein the pre-determined value includes a budget for the item orthe category of the item.
 14. The method of claim 11, wherein thepre-determined value for the item is set by the user.
 15. The method ofclaim 11, wherein the obtaining of the identity and the value of theitem comprises: sending the image data to a server, wherein serverprocesses the image data to determine the identity and the value of theitem; and receiving, from the server, the identity and the value of theitem.
 16. The method of claim 11, wherein prior to receiving the imagedata by the user device: determining a location of the user device; andascertaining that the location of user device is associated with a placethat performs transactions.
 17. The method of claim 11, wherein the userdevice includes any one of a phone, a tablet, and a digital eyeglassdevice.
 18. The method of claim 11, wherein the image includes a movingimage.
 19. A system for performing a real-time overlay on a displayedimage, the system comprising: a mobile device configured to: receive,from a camera, image data of an area comprising an item from thereal-world; send the image data to a server; receive an identity and avalue of the item from the server; determine that the value of the itemis greater than a pre-determined value for the item or a category of theitem; the server configured to: receive, from the mobile device, theimage data of the area comprising the item from the real-world;determine the identity and the value of the item from the image data; adisplay device configured to: display the image of the area comprisingthe item; and obscure, in real-time on the displayed image, the item orcategory of the item having the value that is greater than thepre-determined value.
 20. The system of claim 19, wherein the displaydevice is configured to obscure the item by any one of: obscuring byinserting an opaque polygon into the image data, obscuring by insertinga semi-transparent polygon into the image data, obscuring by pixelatingthe item, and obscuring by reconstructing an inferred background of theitem to give an impression that the item is not there.